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Abstract. We introduce two types of message passing algorithms for 
quantified Boolean formulas (QBF). The first type is a message passing 
based heuristics that can prove unsatisfiability of the QBF by assign- 
ing the universal variables in such a way that the remaining formula is 
unsatisfiable. In the second type, we use message passing to guide branch- 
ing heuristics of a Davis-Putnam Logemann-Loveland (DPLL) complete 
solver. Numerical experiments show that on random QBFs our branch- 
ing heuristics gives robust exponential efficiency gain with respect to the 
state-of-art solvers. We also manage to solve some previously unsolved 
benchmarks from the QBFLIB library. Apart from this our study sheds 
light on using message passing in small systems and as subroutines in 
complete solvers. 



1 introduction 

Satisfiability of quantified Boolean formulas (QBF) is a generalization of the 
Boolean satisfiability problem (SAT) where universal quantifiers are added to the 
existential ones. QBF are useful in modeling practical problems harder than NP 
e.g. planning, verification or combinatorial game playing. Algorithmic complexity 
of QBF ranges in the polynomial hierarchy up to PSPACE. 

Message passing algorithms are used in a wide range of algorithmically hard 
problems, from constraint satisfaction problems such as the satisfiability problem 
[T] to gene regulation network reconstruction [2 , error correcting codes [3], or 
compressed sensing ^^j . Message passing algorithms can be very efficient for large 
random systems where complete algorithms cannot be applied. However, on 
small systems sizes or for structured SAT problems the use of message passing 
has been so far very limited. For instance there have been attempts to solve 
small satisfiability problems by using message passing in a complete algorithm 
[S]. However, according to these result it seems that message passing is helpful 
only for satisfiable random formulas, where complete algorithms are less efficient 
than stochastic local search algorithms. 

In this paper we introduce two types of message passing algorithms for quan- 
tified Boolean Formulas, one as a heuristic scheme and another as a subroutine 
for a complete solver. We mainly consider QBF with two alternations \IX3Y<1> 
with X corresponding to the set of universal variables, Y corresponding to the 
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set of existential variables, and ^ being a CNF formula. Such QBF are said to 
be satisfiable if for all configurations of the universal variables, there exists an 
assignment of the existential variables that satisfies the formula. Examples for 
the case with more alternations are presented in Sec. [5j 

Solving a QBF is in general more difficult than solving a SAT formula. In 
case of random SAT formulas, the state-of-art SAT solvers e.g. kcnfs [6| or march 
[7] can solve hard satisfiable 3-SAT instances with 600 variables, while the state- 
of-art QBF solvers e.g. QuBE7.2 [5] can solve hard random QBF with number 
of variables only up to 60. The intuitive reason for this difference is that a 
much larger search space needs to be explored in QBF. Another reason is that, 
whereas for SAT formulas there are several good decision heuristics based on 
e.g. look-ahead or backbone-search [TU], for QBF efficient decision heuristics 
are missing. Here we introduce belief propagation based decision heuristics that 
provides considerable speed up to the state of the art QBF solvers. A similar 
attempt has been done in where, however, the use of survey propagation 
instead of belief propagation was unfortunate as we discuss briefly later. 

For the SAT problem, besides complete algorithms based on DPLL [12113) 
or resolution, there are also many efficient heuristic algorithms T4i . However up 
to our best knowledge, all algorithms proposed for QBF were complete except 
one [TS]. Here we introduce a new message passing based heuristics for proving 
unsatisfiability that improves over the algorithm of [ T5] . 

2 Definitions and formulas 

In this section we remind the definition of random QBF formulas that we use 
as benchmarks for our algorithms [16117) . We also remind the standard formulas 
for belief propagation and survey propagation update rules [1118119) . 

To express QBF with t alternations, we use QiViQ2V2.-.QtVt<l> where Qn 
denotes quantifier 3 or V at nth alternation, V„ denotes the set of variables at 
nth alternation and <P denotes the set of clauses. We evaluate satisfiability of 
QBF in the following way. If one is able to find an assignment of the universal 
variables for which no solution exists, the QBF is said unsatisfiable, otherwise 
it is satisfiable. For instance, \/X3Y(I> denotes QBF with two alternations, and 
the set of universal X and existential Y variables. This formula is satisfiable if 
for every assignments of X, one can find an assignment for Y such that 'P is 
satisfied. We will use the notation = \X\, = \Y\, M for the number of 
clauses in ^, = M/Ne and Q!„ = M/N^. 

Several models for random QBF were proposed. In this paper, we consider 
the {L,K) model [T7], and the model-B [TB]. In model-B, each clause in <l> has 
U universal variables and V > existential variables that are selected randomly 
from the whole set of universal (resp. existential) variables. The (L, K) model is 
a special case of model-B formula with 2 alternations, which specifies a formula 
\/X3Y<P where each clause in (p contains L + K variables, L from X and K from 
Y. 
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In message passing algorithms, belief propagation |18| and survey propaga- 
tion [T], we define {ipf} to be the marginal probability that variable i takes 
assignment s among all the solutions (BP) or among all the solution clusters 
(SP). In BP s has two possible values + or — , with V'i^ + = 1, in SP s has 
three possible values +,*, and — with tp^ + ^p* + — 1. We say that a variable 
is biased if tp^' ^ ip~ , the larger the difference the larger bias the variable has. 
If tpl' > ip~ we define the bias of i to be , and if ip^' < tp~ we define the bias 
of i to be ^p~ . 

Let us define d+i as the set of clauses to which i belongs non-negated, and 
d-i as the set of clauses to which i belongs negated. Then the set of clauses to 
which variable i belongs can be written as di = {a} U Sia U Uia where (a) if i is 
not negated in a then Sia = d+i \ a, Uia = d-i, and (b) if i is negated in a then 
Sta = d-i \ a, Uia = d+i. 

The BP marginals are computed as 



, (1) 



where messages Ub-^i are a fixed point of the following iterative equations 
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Iteration equations of SP are written as: 
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where Ci^a is a normalization constant ensuring V'i-^a + ^i^a + ^i^a = 1- SP 
marginals are computed by 
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where Ci is again a normalization constant. 



3 Heuristic algorithm for proving unsatisfiability 

Proving unsatisfiability for the two-level QBE can be done by finding an assign- 
ment of the universal variables that leaves the existential part of the formula 
unsatisfiable. One strategy is to find a configuration of the universal variables 
that leaves the largest possible number of clauses unsatisfied; see e.g. WalkMin- 
QBF [TS]. This, however, leads to heuristic algorithms that do not use in any 
way the existential part of the QBE. Here we suggest and test a belief propaga- 
tion decimation heuristics for proving unsatisfiability that takes into account the 
whole formula and outperforms significantly the previously known heuristics. 

Our belief propagation decimation heuristics (BPDU) for proving unsatisfia- 
bility of QBE works as follows: Input is the QBE formula \/X3Y<P. We disregard 
for a moment the quantifiers in the QBE formula and run randomly initialized 
BP on the whole formula (i.e. with both universal and existential variables) till 
convergence or till the maximum number of allowed iterations tmax is achieved 
(typically we use t„iax = 300 in our algorithms). We select the most biased uni- 
versal variable and fix it against the direction of the bias. We repeat the above 
steps until all the universal variables are assigned. Then we run a complete SAT 
solver (in our case kcnfs [6110] ) on the formula consisting of the existential vari- 
ables and clauses that were not satisfied by any of the universal variables. If this 
remaining formula is unsatisfiable then we proved unsatisfiability of the QBE. 
If the remaining formula is satisfiable then the algorithm outputs "unknown", 
since there might be another configuration of the universal variables that would 
leave the remaining formula unsatisfiable. 

A variation on the above heuristic algorithm is to run survey propagation 
instead of belief propagation whenever survey propagation converges to a non- 
trivial fixed point. In what follows we call this variation BPSPDU. 

To evaluate the performance of the BPDU and BPSPDU heuristics we first 
apply them on random (1,2) and (1,3) QBE instances with = Nu = 200 
variables and with varying number of clauses (more clauses make the formulas 
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less likely to be satisfiable), see Fig. [T] For the (1,2) case, a complete solver 
(e.g. QuBe7) can solve every formula hence we can compare the fraction of 
unsatisfiable formulas found by BPDU to the true fraction. For the (1,3) case 
the size is prohibitive for the use of a complete QBF solver. In both cases we 
also show the fraction of unsatisfiable formulas found by the "greedy" strategy, 
which fixes the universal variables in order to let the largest possible number 
of unsatisfied clauses. In case of {l,K) QBF this is easy as every universal 
variable just needs to be set positive if it appears negated in more clauses than 
non-negated, and vice versa. In Fig. [1] we see that the BPDU and BPSPDU 
heuristics perform much better than the greedy strategy. And in case of (1,2) 
QBF the performance is not too far from the optimal. 




Fig. 1. Ratio of unsatisfiable formulas found by the various algorithms discussed in 
the text for random (1,2) QBF instances (left) and (1,3) QBF instances (right) with 
Nu = Ne = 200 as a function of Oe = M /N^. 



In Fig. [5] we compare the performance of the BPSPDU and WalkminQBF 
heuristics for random (2, 3) instances. The WalkMinQBF aims at setting the 
universal variables in order to maximize the number of unsatisfied clauses, and 
then evaluates the remaining SAT formula with a complete SAT solver. Note 
that the complete SAT solver used by WalkminQBF and BPSPDU is the same, 
kcnfs [6110] , so the difference in performance comes only from the quality of the 
universal configuration given by the two heuristics. 

As the figure shows, formulas with larger number of clauses are easier for 
WalkMinQBF, and the running time of the algorithm has larger fiuctuations 
compared to that of BPSPDU. The difference between the two algorithms is 
clearer for larger problem instances; with — Nu = 350, BPSPDU solves 49 
out of the 50 instances, and WalkMinQBF solves only 17 of them within 1000 
seconds. The above results indicate that the universal configuration suggested 
by BPSPDU is much better than the one suggested by WalkMinQBF. 
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Fig. 2. Running time of the BPSPDU heuristics compared to WalkMinQBF on 50 
random (2, 3) QBF benchmarks with the same number of variables — Nu ~ 300 
but different Oe values (left), and the same ae — 11.0 but different number of variable 
Ne = Nu (right) 



4 Message passing to guide QBF complete solvers 

When the BPDU or the BPSPDU algorithms introduced in the last section out- 
put " unknown" , there might be another configuration of the universal variables 
that makes the formula unsatisfiable. Given that we were fixing the universal 
variables starting with the most biased one, it might be a good strategy to back- 
track on the variables fixed in the later stages. In this section we extend this idea 
into a complete DPLL-style solver, which is using message passing to decide on 
which variables to branch next. 

DPLL-style algorithms are the most efficient complete solvers for SAT and 
QBF, they search the whole configurational space by backtracking. The differ- 
ence between DPLL for SAT and for QBF is that in QBF DPLL does back- 
tracking on existential variable when it encounters a contradiction, and does 
backtracking on universal variable when it encounters a solution, see e.g. [30] 
for details. Besides the basic DPLL backtracking procedure, there are several 
components in modern SAT and QBF solvers that lead to exponential speed up, 
among the important ones are decision heuristics, unit-clause propagation, non- 
chronological back-jumping, confiict and solution driven clause learning [20122] . 
Our contribution concerns the decision heuristics which is used in order to de- 
cide which variable will be used in the next branch and which sign of the variable 
should be checked first. Decision heuristics guides DPLL to the more relevant 
branches and keeps it away from irrelevant branches. 

Here we propose a decision heuristics that uses information coming from the 
result of belief propagation (that was iterated till convergence or for imax steps 
on the whole formula ignoring the quantifiers). We propose to start branching 
with the more biased universal variables and assign them first the less probable 
values. For the existential variables we start branching also on the more biased 
ones, but assign them the more probable values. The motivation is that this will 
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speed up the search of a universal configuration that will leave the existential 
part of the QBF unsatisfiable, and the search for a solution on the existential 
part. In particular we propose two decision heuristics. 

In the BPH decision heuristics we simply order the variables according to 
their bias, starting with the most biased one, and assign them first the value 
opposite to the bias for the universal variables and according to the bias for the 
existential ones. 

In the BPDH decision heuristics we run BP and choose the most biased 
unassigned variables which belong to the highest quantifier order. If this variable 
is universal we assign it the value opposite to its bias, if this variable is existential 
we assign it according to its bias. We repeat BP on the simplified formula. 
Finally the BPDH heuristics will branch variables in the same order as they 
were encountered in this procedure. Most of the good decision heuristics for 
SAT and QBF solvers are dynamic, which means that the branching sequence 
is being updated during the run. Our BPH and BPDH decision heuristics are 
computationally heavier than the other efficient decision heuristics e.g. VSIDS, 
MOMs or failed- literal-detection 9^ , so we use it only once at the very beginning 
of the DPLL run. 

We report performance of DPLL with our BPH and BPDH decision heuristics 
on two levels. First level is using BPH and BPDH in the pure DPLL (no features 
such as conflict and solution driving back-jumping and clause learning included). 
The second level is using BPH in a state-of-art QBF solver QuBE7.2, which is 
one of the fastest known solvers today. 

In Fig. [3] we plot the number of solutions and the number of conflicts encoun- 
tered in DPLL using a pure DPLL algorithm and those encountered in DPLL 
with BPDH and BPH decision heuristics on random (1,3) QBF formulas. By 
pure DPLL we mean with no back-jumping nor clause learning, and the de- 
fault decision heuristics is VSIDS (Variable State Independent Decaying Sum) 
[21122] . which is based on dynamic statistics of literal count as a score to or- 
der literals (in case of pure DPLL, no learned clause contributes to the literal 
count). Since pure DPLL is very CPU-time demanding, we use formulas with 
only Nu = Ne = 15 variables. We also plot the ratio of satisfiable formulas Ps, 
and we can see that when almost all formulas are satisfiable, the average number 
of solutions encountered in DPLL is always 2^^, because to prove the satisfia- 
bility of a formula, pure DPLL has to scan all the 2^" universal configurations. 
When Ps becomes smaller than one, DPLL with BPDH and BPH encounters 
much smaller number of solutions than the pure DPLL. Fig. |3] shows that in 
the whole range of parameters the number of conflicts encountered by DPLL 
with BPDH and BPH is always much smaller than DPLL with VSIDS decision 
heuristics. The fewer solutions and conflicts encountered, the smaller search tree 
is explored by the algorithm. 

A better way to extract the full power of BPH and BPDH in DPLL-based 
search is to use solution and conflict driven back-jumping and clause learning. 
In clause learning, reasons of solutions and conflicts are analyzed and stored as 
learned clauses in order to implement non-chronological back-jumping to more 
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Fig. 3. Number of solutions, number of conflicts, running time, and ratio of satisfiable 
formulas Ps, as resulting from pure DPLL for solving random (1, 3) formulas, data are 
averaged over 10000 instances. A''„ = 15 and Ne — 15. 



relevant branches of the search tree and to avoid the encounter of the same 
solutions or conflicts in the future search. With clause learning, DPLL does 
not have to explore 2^" satisfiable leaves of the search tree, and good decision 
heuristics could lead to a smaller number of solutions. We applied clause learning 
to the pure DPLL virith and without BPH. Results show that with BPH and 
BPDH, both search tree size and running time are exponentially smaller than 
without BPH for both unsatisfiable and satisfiable formulas. 

As a next step we implemented message passing decision heuristics in a state- 
of-art solver, we chose QuBE7.2, which uses solution and conflict driven clause 
learning, as the fastest known solver today, and replaced the decision heuristics 
in QuBE7.2 by BPH. We have also tried using BPH in other state-of-art solvers, 
and found quantitatively similar results. 

Our results are presented in Fig. The power of QuBE enables to reach 
larger formulas than we used in Fig. [31 so our experiments are carried out on 
random (1,3) formulas with = iV„ = 50. From the figures we can see that 
BPH considerably reduces the size of the search tree as well as the computation 
time exponentially for whole range of ae- The improvement in performance is 
relatively small only close to the transition region because information given 
by BP is probably less reliable there. Figure [5] shows the computational time 
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Fig. 4. Number of conflicts, solutions and time used by DPLL of QuBE7.2 and 
QuBE7.2+BPH in solving random (1, 3) formulas, data are averaged over 200 instances. 
Nu = 50 and = 50. 



reduction with the system size. We see that with the same time limit, BPH 
enables QuBE to solve larger formulas. 

Ideas similar to ours were already explored in where the authors stud- 
ied an algorithm named HSPQBF that uses Survey Propagation (SP) as deci- 
sion heuristics in a QBF solver QuafHe j20]. We see that using BP as decision 
heuristics is more stable than SP because SP has a narrow region of working 
parameter. Recall that in random SAT formulas with number of variables going 
to infinity, BP reports correct marginals with constraint density ranging from 
zero to the condensation transition point |24j . However, SP always has trivial 
solution (zero messages) when the constrain density is smaller than a value that 
lies relatively close to the SAT-UNSAT transition point [1I25|. Moreover, hard 
QBF instances are often located in the region where SAT formula created by 
ignoring the quantifiers is easy and SP often has trivial solution, but BP works 
well. We cross-checked these intuitions using BPH to guide QuafHe, and com- 
pared running time of DPLL in solving random (1,3) instances by BPH guided 
Quaffle and HSPQBF in Fig. [51 The figure indicates that in the whole range of 
ae, Quaffle with BPH gives better results than HSPQBF. We also checked other 
types of random formulas, similar results are obtained. 
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Fig. 5. Time used by DPLL of QuBE7.2 and QuBE7.2+BPH in solving random (1, 3) 
formulas with fixed Oe values and different number of variables n {Nu — Ne = n). Data 
are averaged over 2000 instances. As shown in Fig.|4l with = 4.0, most formulas are 
satisfiable and with — 5.8, most formulas are unsatisfiable. 

5 Generalization to QBF with multiple alternations 

Our BPH decision heuristics for DPLL works naturally in QBF with muhiple 
alternations. To test performance of BPH in general QBF, we tested model-B 
formulas with 4 alternations, results are plotted in Fig. [T] As the figure shows, 
for small ae BPH improves the performance considerably. However, it gives only 
small performance improvement for large ae when all formulas are unsatisfiable. 

A multi-alteration QBF can be transformed to a 2-alternation QBF by chang- 
ing the order of universal and existential variables. For example, given a 4- 
alternation QBF T = \fXi3X2yX33X4<l>, we can arrive at J"' = 
by switching the order of X2 and X^. One can prove that if J-' is unsatisfi- 
able, then J- is unsatisfiable. The heuristic algorithm can be used to prove the 
unsatisfiability of J^' , and so that of J-. 

6 Performance of BPH on structured formulas 

In contrast with random formulas, BP usually does not give accurate informa- 
tion about the solution space on structured formulas because of existence of 
many short loops. Hence we do not expect BP to improve complete solvers in 
solving every structured formulas. We tested some structured benchmarks from 
QBFLIB py, part of the results are listed in Table 1. We can see from the ta- 
ble that on some formulas, BPH increases the performance of QuBE7.2 while in 
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Fig. 6. Time used by Quaffie+BPH and by HSPQBF [TT] in solving random (1, 3) 
formulas, data are averaged over 2000 instances. Nu ~ 30 and A^e = 30. 



other formulas, BPH decreases the performance. Remarkably, some instances e.g. 
ncf-16-64-8-edau.8 problem and ii8cl-50 problem, which have not been solved 
by other solvers (in previous QBF Evaluations), can be solved by QubE7.2+BPH 
in few seconds. 



Table 1. Running time (in seconds) of QuBE7.2 and QuBE7.2+BPH in solving struc- 
tured benchmarks. 



Name of instance 


QuBE 


QuBE+BPH 


Name of instance 


QuBE 


QuBE+BPH 


ii8cl-50 


> 600 


14.21 


ev-pr-8x8-15-7-0-l-2-lg 


0.023 


> 600 


ncf_16_128_2_edau.8 


> 600 


0.27 


flipfiop-12-c 


0.71 


> 600 


ncf_16_64_8_euad.4 


> 600 


4.01 


lut4_3_fAND 


0.25 


> 600 


X170.5 


520.44 


63.42 


cf_2_9x9_w_ 


0.18 


539.49 


ncf_8_32_4_euad.4 


301.86 


0.14 


ncf_4_32_4_u.9 


35.69 


565.35 


szymanski-5-s 


153.00 


19.31 


c2_BMC_p2_k8 


95.49 


207.25 


k_d4_n-5 


408.15 


83.94 


toilet_a_10_01.15 


7.45 


510.84 


ncf_4_32_8_edau.3 


349.78 


0.25 


cf_3_9x9_d_ 


0.17 


480.27 


X170.19 


340.05 


13.07 


connect _9x8_8_D 


0.08 


115.79 


k_grz_p-13 


190.66 


24.99 


stmt21_252_267 


3.93 


142.49 


connect _5x4_3_R 


97.51 


38.35 


k_ph_n-20 


7.65 


462.44 


BLOCKS4i.6.4 


49.21 


29.87 


toilet_a_10_01.11 


0.02 


416.16 


C880.blif_0.10_l. 00 












_0_l_out_exact 


93.21 


44.45 


xll5.4 


201.46 


302.64 


CHAIN18V.19 


346.39 


288.12 


stmt21_143_314 


92.61 


92.75 
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Fig. 7. Number of conflicts, solutions and time used by DPLL of QuBE7.2 in solving 
model-B formulas, data are averaged over 200 instances. Formulas have 4 alternations 
and n = 20 variables in each alternation. Each clause in the formula contains 4 exis- 
tential variables that are selected randomly from all the existential variables (i.e. from 
the 2nd and 4th level variables) and 1 universal variable that is selected randomly 
from all the universal variables (i.e. from the 1st and 3rd level variables). Oe in figure 
corresponds to number of clauses divided by n. In generating formula, we use formula 
generator downloaded from QBFLIB [23] . 



7 Conclusion and discussion 



In this paper we developed heuristic and complete algorithms for QBF based on 
message passing. Our heuristic algorithm BPDU and BPSPDU use message pass- 
ing to find a universal assignment that evaluates to an unsatisfiable remaining 
formula, thus to prove the unsatisflability of 2-alternation QBF. Our complete 
algorithm is based on DPLL process, which searches the whole configurational 
space by backtracking more efficiently using message passing branching heuris- 
tics BPDH and BPH. Our algorithms described above can be downloaded from 
[26] . Both our algorithms work very well on random QBF and in some cases 
they provide large improvement also on structured QBF and solve some previ- 
ously unsolved benchmarks in QBFLIB. These results should encourage further 
investigation of the use of message passing as heuristic solvers or as guides for 
heuristics included in DPLL-like complete solvers. 
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